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1. DETAILED ACTION 
2. Continued Examination Under 37 CFR 1. 1 14 

3. A request for continued examination under 37 CFR 1.114, including tine fee set 
forth in 37 CFR 1 .1 7(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
201 1/03/22 has been entered. 

4. Claim Objections 

5. Claim 22, 23, 24 objected to because of the following informalities: claim 22, 23, 
24 depend on non-existent claim 6, the examiner will assume that claim 22, 23, 24 
depends on claim 21 for examination purposes. Appropriate correction is required. 

6. Ciaim Rejections - 35 USC § 103 

7. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

8. A patent may not be obtained though the invention is not identically disclosed or described as set forth 
in section 1 02 of this title, if the differences between the subject matter sought to be patented and the prior art 
are such that the subject matter as a whole would have been obvious at the time the invention was made to a 
person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived 
by the manner in which the invention was made. 
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9. Claim 16, 17, 18, 20, 21, 22, 23, 24 rejected under 35 U.S.C. 103(a) as being 
unpatentable over Konrad (US-5974444), and further in view of Serbanescu (US- 
20030162595). 

1 0. As to claim 16,21: Konrad teaches a method for processing video/audio data 
synchronized with sensory data, comprising the steps of: receiving multiplex data 
(fig.3c and 7:16-19, 39-45, 10:12-18: a server performs multiplexing and de- 
multiplexing of data) including video/audio data and sensory data in a predetermined 
form where the video/audio data is presented together W\th the sensory data (fig.1, 10 
and 9:1-59, 13:45-49, 21 :1-45, ESP 9:30-40, 50-59: convey data from computer to 
human user including sound, tactile, touch, vision, taste or smell), wherein the 
sensory data describes at least one of a touch, an odor and a taste which is to be 
expressed with the video/audio data (21 :35-44: video, audio, olfactory, tactile, and 
gustatory data presented together through human interface service); 
demultiplexing the multiplex data (fig.3c and 7:16-19, 39-45, 10:12-18: a server 
performs multiplexing and de-multiplexing of data) into the video/audio data and the 
sensory data; and outputting the video/audio data presented together W\th at least one 
of the touch, the odor and the taste according to the sensory data, wherein the touch, 
the odor and the taste are outputted through a touch device, an odor device and a taste 
device, respectively (21:35-44: video, audio, olfactory, tactile, and gustatory data 
presented together through human interface service; 9:30-40 and 9:50-59: output 
portions). 
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1 1 . Konrad may not explicitly teach synchronizing video / audio data with sensory 
data. However, Serbanescu teaches synchronizing video / audio data with sensory 
data ([0005, 0008, 0030, 0059-0061]). 

1 2. Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to implement the teachings of Serbanescu into Konrad since 
Konrad suggests presenting video, audio, olfactory, tactile, and gustatory together to the 
user (21 :35-44) In general and Serbanescu suggests synchronizing output media, such 
as taste, odor, touch along with audio and video, the motivation being to present an 
interactive media experience involving all the senses ([0005, 0008, 0030, 0059-0061]). 

1 3. As to claim 1 7: Konrad teaches the method as recited in claim 1 , further 
comprising the step of: transmitting the multiplex data to a receiver (fig.3c, 102, 110 
and 7:16-19, 39-45, 10:12-18: a server performs multiplexing and de-multiplexing 
of data). 

14. As to claim 18, 22: Konrad teaches the method as recited in claim 1 , 6, wherein 
the sensory data describing the touch includes information on at least one of whether 
the touch is described, whether right/left movement is described, whether up/down 
movement Is described, whether back/forth movement Is described, a distance of 
movement, a speed of movement, an acceleration of movement, whether right/left 
rotation is described, an angle of right/left rotation, a speed of right/left rotation, and an 
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acceleration of right/left rotation (9:50-59: touch is described by pressure or 
temperature). 

1 5. As to claim 19, 23: Konrad teaches the method as recited in claim 1 , 6, wherein 
the sensory data describing the odor includes information on at least one of whether the 
odor is described, a kind of the odor, and an intensity of the odor (13:45-49: olfactory 
expression, i.e. description). 

1 6. As to claim 20, 24: Konrad teaches the method as recited in claim 1 , 6, wherein 
the sensory data describing the taste includes information on at least one of whether the 
taste is described, a kind of the taste, and an intensity of the taste (13:45-49: gustatory 
expression, i.e. description). 

17. Conclusion 

1 8. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to ANDREW OH whose telephone number is (571 )270- 
5273. The examiner can normally be reached on M-F 8:30AM - 5AM EST. 

1 9. If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Daniel J. Ryman can be reached on (571)272-3152. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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20. Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

21. 22. 

23. /A. 0.1 

24. Examiner, Art Unit 2466 
/Daniel J Ryman/ 

Supervisory Patent Examiner, Art Unit 2466 



